Schema Evolution and Gravitation to Rigidity: A Tale of Calmness in the Lives of Structured Data

Evolving dependency magnets, i.e., software modules upon which a large number of other modules depend, is always a hard task. As Robert C. Martin has nicely summarized it (see http://www.oodesign.com/design-principles.html), fundamental problems of bad design that hinder evolution include immobility, i.e., difficulty in reuse, rigidity, i.e., the tendency for software to be difficult to change and fragility, i.e., the tendency of the software to break in many places every time it is changed. In such cases, developers are reluctant to evolve the software to avoid facing the impact of change. How are these fundamentals related to schema evolution? We know that changes in the schema of a database affect a large (and not necessarily traced) number of surrounding applications, without explicit identification of the impact. These affected applications can then suffer from syntactic and semantic inconsistencies – with syntactic inconsistency leading to application crashes and semantic inconsistency leading to the retrieval of data other than the ones originally intended. Thus, the puzzle of gracefully facilitating the evolution of data-intensive information systems is evident, and the desideratum of coming up with engineering methods that allow us to design information systems with a view to minimizing the impact of evolution, a noble goal for the research community. Several research paths towards this goal are being pursued. A first path involves works concerning an algebra of schema evolution operations, that can allow the description of the history of schema changes in a semantically rich sequence of operations [4,7]. Another path involves the management of the impact of changes [1,9]. A fairly novel path involves the identification of profiles and patterns in the usage of relational database access technologies in open source projects [5,6,10]. Moreover -and in particular, what interests us in the context of this talkas all engineering should be based on well-understood mechanics and laws, the research community has also tried to uncover mechanics and patterns that govern schema evolution. Knowing the underlying mechanisms of schema evolution is fundamental in engineering solutions that gracefully handle it: without this knowledge we can easily stray in solutions that have no relationship to real-world problems. Although the body of work is rather small compared to the importance of the matter, one cannot ignore that access to schema histories was practically impossible before the proliferation of Free and Open Source Software (FOSS). Thus, apart from an original study in the early ’90s [12], it was